Machine dump body control using object detection

ABSTRACT

A mobile work machine includes a frame, a material loading system having a material receiving area configured to receive material and an actuator configured to control the material loading system to move the material receiving area relative to the frame, and a control system configured to receive an indication of a detected object, determine a location of the object relative to the material loading system, and generate a control signal that controls the mobile work machine based on the determined location.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of and claims priority of U.S.patent application Ser. No. 16/803,603, filed Feb. 27, 2020, the contentof which is hereby incorporated by reference in its entirety.

FIELD OF THE DESCRIPTION

The present description generally relates to object detection andcontrol systems for mobile work machines. More specifically, but not bylimitation, the present description relates to dump body control on amobile work machine using object detection.

BACKGROUND

There are many different types of work machines. Those work machines caninclude construction machines, turf management machines, forestrymachines, agricultural machines, among others. Many of these pieces ofmobile equipment have controllable subsystems, that include mechanismsthat are controlled by the operator in performing operations.

For instance, a construction machine can have multiple differentmechanical, electrical, hydraulic, pneumatic and electro-mechanicalsubsystems, among others, all of which can be operated by the operator.Construction machines are often tasked with transporting material acrossa worksite, or into or out of a worksite, in accordance with a worksiteoperation. Different worksite operations may include moving materialfrom one location to another or leveling a worksite, etc. During aworksite operation, a variety of construction machines may be used,including articulated dump trucks, wheel loaders, excavators, boomlifts, among others.

The discussion above is merely provided for general backgroundinformation and is not intended to be used as an aid in determining thescope of the claimed subject matter.

SUMMARY

A mobile work machine includes a frame, a material loading system havinga material receiving area configured to receive material and an actuatorconfigured to control the material loading system to move the materialreceiving area relative to the frame, and a control system configured toreceive an indication of a detected object, determine a location of theobject relative to the material loading system, and generate a controlsignal that controls the mobile work machine based on the determinedlocation.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter. The claimed subject matter is not limited to implementationsthat solve any or all disadvantages noted in the background.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing one example of a work machinearchitecture that includes a mobile work machine.

FIG. 2 is a pictorial illustration showing one example of a mobile workmachine.

FIG. 3 is a pictorial illustration showing one example of a mobile workmachine.

FIG. 4 is a pictorial illustration showing one example of a mobile workmachine.

FIG. 5 is a block diagram illustrating one example of an objectdetection system.

FIG. 6 is a flow diagram illustrating an example operation of an objectdetection system.

FIG. 7 shows an example of a mobile device that can be used in thearchitecture(s) shown in the previous figures.

FIG. 8 is a block diagram showing one example of a computing environmentthat can be used in the architecture(s) shown in the previous figures.

DETAILED DESCRIPTION

Mobile work machines operate on worksites that are often complicated,sometimes with many other work machines and workers performing manydifferent operations at any given time. Worksite operations may involvea large number of steps or phases and may be quite complex. Further, theworksite operations often require precise machine control by anoperator, and this machine control can be highly repetitive. To furthercomplicate matters, a typical worksite comprises a variety of differenthazards or obstacles, generally referred to herein as objects.

The more complicated a worksite gets, the more difficult it can be foran operator of a work machine to complete their task while avoidingcollisions with objects, such as another work machine, an above or belowground hazard, a person at the worksite or any other object. Forinstance, a material loading system of a mobile work machine includes amaterial receiving area (e.g., a dump body, such as a bucket, container,etc. on a construction machine) that is movable by a dumping actuator tounload the material receiving area. For this dumping operation, anoverhead hazard can include an overhead utility, aerial vehicle, drone,building overhang, etc. In many cases, such objects are not easilyidentifiable by the operator, especially when they are above and/orbehind the operator.

The present description generally relates to object detection andcontrol systems for mobile work machines. More specifically, but not bylimitation, the present description relates to dump body control on amobile work machine using object detection.

FIG. 1 is a block diagram showing one example of a work machinearchitecture 100 that includes a mobile work machine 102 (such as aconstruction machine, examples of which are discussed below). Workmachine 102 includes a control system 104 configured to control a set ofcontrollable subsystems 106 that perform operations on a worksite. Forinstance, an operator 108 can interact with and control work machine 102through operator interface mechanism(s) 110. Operator interfacemechanism(s) 110 can include such things as a steering wheel, pedals,levers, joysticks, buttons, dials, linkages, etc. In addition, they caninclude a display device that displays user actuatable elements, such asicons, links, buttons, etc. Where the device is a touch sensitivedisplay, those user actuatable items can be actuated by touch gestures.Similarly, where mechanism(s) 110 includes speech processing mechanisms,then operator 108 can provide inputs and receive outputs through amicrophone and speaker, respectively. Operator interface mechanism(s)110 can include any of a wide variety of other audio, visual or hapticmechanisms.

Work machine 102 includes a communication system 112 configured tocommunicate with other systems or machines in architecture 100. Forexample, communication system 112 can communicate with other localmachines, such as other machines operating on a same worksite as workmachine 102. In the illustrated example, communication system 112 isconfigured to communicate with one or more remote systems 114 over anetwork 116. Network 116 can be any of a wide variety of different typesof networks. For instance, it can be a wide area network, a local areanetwork, a near field communication network, a cellular communicationnetwork, or any of a wide variety of other networks, or combinations ofnetworks.

A remote user 118 is illustrated as interacting with remote system 114,such as to receive communications from or send communications to workmachine 102 through communication system 112. For example, but not bylimitation, remote user 118 can receive communications, such asnotifications, requests for assistance, etc., from work machine 102 on amobile device.

FIG. 1 also shows that work machine 102 includes one or more processors122, one or more sensors 124, an object detection system 126, a datastore 128, and can include other items 130 as well. Sensor(s) 124 caninclude any of a wide variety of sensors depending on the type of workmachine 102. For instance, sensors 124 can include object detectionsensor(s) 132, material sensors 134, position/route sensors 136, speedsensors 138, worksite imaging sensors 140, and can include other sensors142 as well.

Material sensors 134 are configured to sense material being moved,processed, or otherwise worked on by work machine 102. Position/routesensors 136 are configured to identify a position of work machine 102and a corresponding route (e.g., heading) of work machine 102 as ittraverses the worksite. Sensors 136 include sensors configured togenerate signals indicative of an angle or turn radius of machine 102.This can include, but is not limited to, steering angle sensors,articulation angle sensors, wheel speed sensors, differential drivesignals, gyroscopes, to name a few.

Speed sensors 138 are configured to output a signal indicative of aspeed of work machine 102. Worksite imaging sensors 140 are configuredto obtain images of the worksite, which can be processed to identifyobjects or conditions of the worksite. Examples of imaging sensor(s) 140include, but are not limited to, one or more cameras (e.g., a monocularcamera, stereo camera, etc.) that obtains still images, a time-series ofimages, and/or video feed of an area of a worksite. For instance, thefield of view (FOV) of the camera can include any area of interest onthe worksite. This can include areas above and/or to the rear of machine102, and which may not otherwise be visible to operator 108 while in theoperator compartment or cab of machine 102.

The camera can include any suitable image acquisition system including,but not limited to, an area array device such as a charge couple device(CCD) or a complementary metal oxide semi-conductor (CMOS) image device.Further, the camera can be coupled to any suitable optical system toincrease or decrease the field of view under control of control system104. Further still, the camera may be provided with additionalillumination, such as a backup light, or dedicated illuminator, suchthat images can easily be acquired when excavator is operated inlow-light conditions. Further still, in one example multiple cameras areused to provide stereo vision. In this way, using stereo visiontechniques, three-dimensional imagery and visual odometry can beemployed.

Object detection sensors 132 can include electromagnetic radiation (EMR)sensors (e.g., transmitters, receivers, transceiver(s)) 144. Examples ofEMR sensors include imaging sensors 140 (discussed above), radiofrequency (RF) devices 146 (such as RADAR), LIDAR devices 148, and caninclude other devices 150 as well. Object detection sensors 132 can alsoinclude sonar devices 152, and can include other devices 154 as well.

Control system 104 interacts with data store 128 (e.g., storing orretrieving data). Data store 128 can store a variety of information onthe worksite and the objects/terrain of the worksite. Illustrativelyshown, data store 128 includes object data 156, can include other dataas well, as indicated by block 158.

Object data 156 includes data related to the various objects in theworksite (e.g., work machine 102, other work machines, operator 108,hazards, etc.). For instance, object data 156 can include position dataidentifying positions of the objects in the worksite. For example, thiscan include the GPS coordinates or local coordinates of the objectsand/or the vertical portion of the object relative to the ground.Position data for the objects can be used by control system 104 to avoidcollisions between objects.

Additionally, object data 156 can include the dimensions of the objectsin the worksite. For example, the physical dimensions of an object. Forinstance, work machine 102, in one example, is thirteen feet long byeight feet wide by nine feet high. Dimensions of objects in the worksitecan be used by control system 104 to prevent collisions between objects.

Also, object data 156 can include pose data of the objects in theworksite. For example, pose data for an object includes positions anddimensions of its components. For instance, pose data for work machine102 can include the positions of various components (e.g., dump body,bucket, etc.) relative to a frame and/or other components. Pose data ofobjects in the worksite can be used by control system 104 to preventcollisions between objects with greater precision than standarddimensions alone. For example, work machine 102 in one pose at a givenlocation may not cause a collision, but work machine 102 in a differentpose at the same location, may cause a collision.

As illustrated, control system 104 includes settings control logic 160,route control logic 162, location logic 163, machine geometry logic 164,collision logic 166, proximity logic 168, display generator logic 170,and it can include other items 172. Controllable subsystems 106 caninclude propulsion system 174, steering system 176, material loading (orhandling) system 178, one or more different actuators 180 (e.g., thatcan be used to change machine settings, machine configuration, etc.),and it can include a wide variety of other systems 182.

As illustrated, material loading system 178 includes a materialreceiving area 184, and one or more actuator(s) 186 configured to move(e.g., load/unload) the material receiving area. Material receiving area184 is configured to receive material (e.g., soil, building materials,etc.), and can take a wide variety of different forms depending on thetype of machine 102. For instance, in the case of a dump vehicle (e.g.,rear or side dump truck), material receiving area 184 comprises apivotable (or otherwise moveable) dump body. In the case of anexcavator, material receiving area 184 comprises a bucket supported onan articulated arm.

In one example, controllable subsystems 106 also include operatorinterface mechanism(s) 110, such as display devices, audio outputdevices, haptic feedback mechanisms, as well as input mechanisms.Examples are discussed in further detail below.

Settings control logic 160 can control one or more of subsystems 106 inorder to change machine settings based upon objects, conditions, and/orcharacteristics of the worksite. By way of example, settings controllogic 160 can actuate actuator(s) 180 and/or 186 that change theoperation of propulsion system 174, steering system 176, and/or materialloading system 178.

Route control logic 162 can control steering system 176. By way ofexample, but not by limitation, if an object is detected by objectdetection system 126, route control logic 162 can control propulsionsystem 174 and/or steering system 176 to avoid the detected object.Location logic 163 determines a location of machine 102 on the worksite.

Machine geometry logic 164 retrieves physical dimensions of work machine102 and its controllable subsystems 106, and sensor signals from sensors124. Machine geometry logic 164 uses this data to determine the pose ofwork machine 102 (e.g., the orientation and positions of work machine102 and all of its controllable subsystems 106). The pose of workmachine 102 can be useful in determining if a portion of work machine102 will collide with another object.

Collision logic 166 is operably or communicatively coupled to settingscontrol logic 160, route control logic 162, location logic 163, machinegeometry logic 164, and/or a source of external object locations togather information used to determine if work machine 102 will collidewith another object. Before control logic 160 and/or 162 sends controlsignals, collision logic 166 can determines if the potential action willcause a collision with an external object and, if so, prevent theactuating signal from being sent. For example, collision logic 166 cansimulate the requested actuation and determine if the actuation willcause an intersection of objects. If so, there will be a collision andthe actuation is prevented.

Similar to collision logic 166, proximity logic 168 is coupled tosettings control logic 160, route control logic 162, location logic 163,machine geometry logic 164, and/or a source of external object locationsto gather information used to determine if work machine 102 will collidewith another object. Proximity logic 168, however, determines that workmachine 102 is within a threshold distance of an external object andprevents work machine 102 from moving within a threshold distance of anobject. The threshold distance can be variable based on the type ofobject. For example, a threshold distance from an overhead power linemay be twenty feet, while the threshold distance to a tree may be fivefeet.

Display generator logic 170 illustratively generates a control signal tocontrol a display device, to generate a user interface display foroperator 108. The display can be an interactive display with user inputmechanisms for interaction by operator 108.

Object detection system 126 is configured to receive signals from objectdetection sensor(s) 132 and, based on those signals, detect objectsproximate machine 102 on the worksite, such as in a path of machine 102.Object detection system 126 can therefore assist operator 108 inavoiding objects while moving machine 102 along the worksite (e.g.,backing up) and/or moving receiving area 184 (e.g., whiletransporting/unloading/dumping material). Before discussing objectdetection system 126 in further detail, examples of mobile work machineswill be discussed with respect to FIGS. 2-4 .

As noted above, mobile work machines can take a wide variety ofdifferent forms. FIG. 2 illustrates one example of a mobile work machine202, in the form of an off-road construction vehicle (illustratively arear dump vehicle or truck). Machine 202 includes a power head section204 and a load carrying section 206. The power head section 204 includesa vehicle engine or motor 208, an operator cab 210 and a front axle andwheels 212 which are all coupled to a front frame 214. The load carryingsection 206 includes a dump body 216, a first rear axle and wheels 218and a second rear axle and wheels 220 which are all coupled to a rearframe 222. The front frame 214 of the power head section 204 is coupledto the rear frame 222 of the load carrying section 206 by articulationand oscillation joints 224. The articulation joint enables the powerhead section 204 and the load carrying section 206 to pivot relative toone another about a vertical axis for steering machine 202, and theoscillation joint allows the power head section 204 and the loadcarrying section 206 to rotate relative to one another about alongitudinal axis extending along the length of machine 202. The dumpbody 216 is illustrated with dashed lines as dump body 216′ when in thedump position.

Machine 202 includes an object detection system 230 (e.g., system 126)and a control system 232 (e.g., system 104). Object detection system 230detects objects located within a range of machine 202. In theillustrated example, system 230 receives signals from object detectionsensor(s) 234 (e.g., sensor(s) 132) which are mounted to detect objectsto a rear and/or above dump body 216. The components of system 230and/or system 232 communicate over a CAN network of machine 202, in oneexample.

Object detection sensor(s) 234 are configured to generate a signalindicative of a detected object (represented generally at block 236) anda location of the detect object relative to dump body 216 (or otherportions of machine 202). For instance, sensor 234 can comprise animaging sensor or camera. Alternatively, or in addition, sensor(s) 234can be configured to transmit a detection signal toward the rear ofmachine 202 and receives reflections of the detection signal to detectobject 236 behind machine 202. In one example, the detection signalcomprises electromagnetic radiation transmitted to the rear of machine200. For instance, this can include radio frequency (RF) signals. Someparticular examples include radar and LORAN, to name a few. In otherexamples, object detection sensor(s) 234 utilize sonar, ultrasound, aswell as light (e.g., LIDAR) to image objects. Example LIDAR systemsutilize ultraviolet light, visible light, and/or near infrared light toimage objects.

Of course, other types of object detectors can be utilized. In any case,object detection system 230 generates outputs indicative of objects,which can be utilized by control system 232 to control operation ofmachine 202.

FIG. 3 illustrates another example of a mobile work machine 302, in theform of an off-road construction vehicle (illustratively a front orwheel loader). Machine 302 includes a cab 304 having a display device306, ground-engaging element(s) 308 (e.g., wheels), motor(s) 310, speedsensor(s) 312, a frame 314, and a boom assembly 316. Boom assembly 316includes a boom 318, a boom cylinder 320, a bucket 322 and a bucketcylinder 324. Boom 318 is pivotally coupled to frame 314 and may beraised and lowered by extending or retracting boom cylinder 320. FIG.illustrates bucket 322 in an unloading or dump position 322′.

Bucket 322 is pivotally coupled to boom 318 and may be moved through anextension or retraction of bucket cylinder 324. During operation, mobilemachine 302 can be controlled by an operator within cab 304 in whichmobile machine 302 can traverse a worksite. In one example, each one ofmotor(s) 310 are illustratively coupled to, and configured to drive,wheel(s) 308 of mobile machine 302. Speed sensor(s) 312 areillustratively coupled to each one of motor(s) 310 to detect a motoroperating speed.

In the illustrated example, machine 302 comprises an articulating bodywhere a front portion 326 is pivotably connected to a rear portion 328at a pivot joint 330. An articulation sensor can be utilized todetermine the articulation angle, at pivot joint 330, which can be usedto determine the path of machine 302. In another example in which thebody of machine 302 is non-articulating, the angle of the front and/orrear wheels 308 is rotatable relative to the frame.

Machine 302 includes an object detection system 332 and a control system334. In one example, systems 332 and 334 are similar to systems 230 and232 illustrated in FIG. 2 . Object detection system 332 detects objectslocated within a range of machine 302. In the illustrated example,system 332 receives signals from object detection sensor(s) 336 (e.g.,sensor(s) 132) which are mounted to detect objects 338 above (or atleast potentially in the path of) bucket 322.

FIG. 4 illustrates another example of a mobile work machine 402, in theform of an off-road construction vehicle (illustratively a hydraulicexcavator). Machine 402 includes a house 404 having an operator cab 406rotatably disposed above tracked portion 408. House 404 may rotatethree-hundred sixty degrees about tracked portion 408 via rotatablecoupling 410. A boom 412 extends from house 404 and can be raised orlowered in the direction indicated by arrow 414 based upon actuation ofhydraulic cylinder 416. A stick 418 is pivotably connected to boom 412via joint 420 and is movable in the direction of arrows 422 based uponactuation of hydraulic cylinder 424. Bucket 426 is pivotably coupled tostick 418 at joint 428 and is rotatable in the direction of arrows 430about joint 428 based on actuation of hydraulic cylinder 432.

When an operator within cab 406 needs to move boom 412, he or sheengages suitable controls and, in one example, can automaticallyactivate a camera which provides a camera image, on a display within cab406, corresponding to field of view of an area in which boom 412 ismoving or will be moving for the operator's commanded movement.

Machine 302 includes an object detection system 434 and a control system436. In one example, systems 434 and 436 are similar to systems 230 and232 illustrated in FIG. 2 . Object detection system 434 detects objectslocated within a range of machine 402. In the illustrated example,system 434 receives signals from object detection sensor(s) 438 (e.g.,sensor(s) 132) which are mounted to detect objects 440 above (or atleast potentially in the path of) boom 412.

FIG. 5 illustrates one example of object detection system 126. For sakeof illustration, but not by limitation, object detection system 126 willbe described in the context of mobile work machine 102 illustrated inFIG. 1 .

System 126 includes initiation logic 502 configured to initiate andcontrol object detection performed by system 126. For example, this canbe in response to a mode selector 504 determining that the machine 102has entered a particular mode for which system 126 is to be initiated.For example, this can be in response to determining that system 178 isbeing moved or otherwise being actuated (or preparing to be moved oractuated), by sensing operator inputs and/or machine settings.

Sensor control logic 506 is configured to control object detectionsensors 132 to detect any objects in a projected path of system 178.This can include controlling imaging sensors (e.g., cameras) 140 toacquire images. In another example, logic 506 controls sensor(s) 144 totransmit detection signals and to receive corresponding reflections ofthe detection signal. The signals from sensors 132 are used by objectdetection logic 508 to detect the presence of objects (e.g., objects236, 338, 440) on the worksite. Object location determination logic 510is configured to determine a location of the object(s) detected by logic508.

In one example, a vision recognition system 512 is configured to performvision recognition on acquired images, to evaluate the objects detectedby logic 508. Illustratively, system 512 includes image processing logic514 is configured to perform image processing on the image and objectevaluation logic 516 is configured to evaluate the object based on theimage processing performed by logic 514. This can include, but is notlimited, object size detection 518, object shape detection 520, objectclassification performed by an object classifier 522, and can includeother items 524 as well.

Path determination logic 526 is configured to determine a path of system178 (e.g., movement of receiving area 184) and/or other components ofmachine 102, and control signal generator logic 528 is configured togenerate control signals, either by itself or in conjunction withcontrol system 104. System 126 is illustrated as having one or moreprocessors 530, and can include other items 532 as well.

FIG. 6 illustrates a flow diagram 600 of an example operation of objectdetection system 126. For sake of illustration, but not by limitation,FIG. 6 will be described in the context of mobile work machine 102 shownin FIG. 1 .

At block 602, initiation logic 502 initiates object detection. This canbe in response to a manual input by operator 108, such as operator 108actuating an input mechanism. This is represented by block 604.Alternatively, or in addition, the object detection can be initiatedautomatically, such as in response to logic 502 detecting that machine102 has entered a particular operating mode, such as being shifted intoreverse, activating/actuating material loading system 178 (e.g., liftingdump body 216, bucket 322, boom 412, etc.). This is represented by block606.

Of course, the object detection can be initiated in other ways as well.This is represented by block 608.

At block 610, sensor control logic 506 controls object detectionsensor(s) 132 to detect objects on the worksite. This can includeobjects that are at or close to the ground level, as well asabove-ground objects such as trees, power lines, building overhangs,building ceilings, etc. This can be done in any of a number of ways. Forexample, images can be obtained from imaging sensor(s) 140. This isrepresented by block 612. In another example, a radar transmitter cantransmit a radar signal, represented by block 614. Alternatively, or inaddition, a detection signal can comprise a LIDAR device. This isrepresented by block 616. Of course, other types of detection can beperformed as well. This is represented by block 618.

Object location determination logic 510 receives the sensor signals (orrepresentations of the sensor signals) that are indicative of thedetected objects and determines locations of each object relative tomachine 102. This is represented by block 620.

In one example of block 620, logic 510 determines a distance of theobject from the sensor mounting location on machine 102. This isrepresented by block 622. For example, in the example of FIG. 2 , system230 determines a distance of object 236 from sensor 234.

Logic 510 also can determine an angle of the detected object relative tothe sensor mounting orientation. This is represented by block 624. Usingthe received information, logic 510 determines the position of theobject relative to machine 102 and/or the worksite itself. Of course,the location of each object can be determined in other ways as well.This is represented by block 626.

Using this information, the location of each detected object can becorrelated to the current position and/or projected movement path ofmaterial loading system 178. This is represented by block 628. Forexample, path determination logic 526 can utilize machine geometry logic164 to determine the path of system 178 for a commanded movement. Thisis represented by block 630. In the example of FIG. 2 , this includeslogic 526 determining the path of dump body 216 as it moves from theloaded position to the unloading or dumping position 216′.

In one example, the correlation can be determined based on the objectposition (block 632), object size (block 634), and/or object shape(block 636). Also, in one example an image classifier is applied todetect and object type. This is represented by block 638. In oneexample, vision recognition system 512 can be configured to apply imageobject classifier 522 to determine whether the detected object is aperson, a utility line (high voltage power line, etc.), a tree, abuilding, etc. In one example, the type of object is utilized to selecta threshold proximity for collision avoidance. As mentioned above, ifthe object is a high voltage power line, the threshold may be set attwenty feet whereas if the object is a tree, the threshold may be set atfive feet. Of course, these are for sake of example only. Of course, thelocation of each object can be correlated to the machine position inother ways as well. This is represented by block 640.

At block 642, each object is evaluated based on a probability ofentering the threshold proximity. Based on this determination, a controlsignal is generated to control the machine at block 644. Machine 102 canbe controlled in any of a number of ways. In one example, one or more ofcontrollable subsystems 106 is controlled by control signal generatorlogic 528 and/or control system 104. This is represented by block 646.

For example, control system 104 can automatically move system 178 toavoid collision with the object. In another example, a thresholdposition of system 178 can be set to avoid collision with the object.For sake of illustration, but not by limitation, in the example of FIG.2 block 646 can set a threshold dump position of seventy-five percent(or otherwise) so that the dump body 216 cannot be raised above a heightthat would result in contact (or entering the threshold distance to)object 236. Of course, this is for sake of example only. In anotherexample, propulsion system 174 and/or steering system 176 can beautomatically controller to avoid contact with the object (e.g.,automatically braking or stopping the wheels, stopping the engine,shifting gears, etc.).

Alternatively, or in addition, operator interface mechanisms can becontroller to render visual, audio, haptic, or other types of outputs tooperator 108, indicative of the detected object(s). This is representedby block 648. For example, a visual warning can be generated on adisplay device to operator 108 to indicate that system 178 isapproaching the detected object. In one example, the visual indicationcan comprise an augmented video feed from the camera showing the objectin the camera field of view. The display can be augmented in any of anumber of ways, such as, but not limited to, highlighting the object onthe display. Alternatively, or in addition, an audible warning can alsobe generated, and can change volume (or otherwise) based on a detecteddistance to the object (i.e., the audible alarm becomes louder asmachine 102 approaches the object). In another example, haptic feedbackin the form of a seat and/or steering wheel vibration can be provided tooperator 108.

Of course, control signal(s) can be generated to control machine 102(and/or other machines/systems) in other ways as well. This isrepresented by block 650. Block 652 determines whether to continueobject detection. This is can be done automatically, manually, orotherwise.

The present discussion has mentioned processors and servers. In oneembodiment, the processors and servers include computer processors withassociated memory and timing circuitry, not separately shown. They arefunctional parts of the systems or devices to which they belong and areactivated by, and facilitate the functionality of the other componentsor items in those systems.

It will be noted that the above discussion has described a variety ofdifferent systems, components and/or logic. It will be appreciated thatsuch systems, components and/or logic can be comprised of hardware items(such as processors and associated memory, or other processingcomponents, some of which are described below) that perform thefunctions associated with those systems, components and/or logic. Inaddition, the systems, components and/or logic can be comprised ofsoftware that is loaded into a memory and is subsequently executed by aprocessor or server, or other computing component, as described below.The systems, components and/or logic can also be comprised of differentcombinations of hardware, software, firmware, etc., some examples ofwhich are described below. These are only some examples of differentstructures that can be used to form the systems, components and/or logicdescribed above. Other structures can be used as well.

Also, a number of user interface displays have been discussed. They cantake a wide variety of different forms and can have a wide variety ofdifferent user actuatable input mechanisms disposed thereon. Forinstance, the user actuatable input mechanisms can be text boxes, checkboxes, icons, links, drop-down menus, search boxes, etc. They can alsobe actuated in a wide variety of different ways. For instance, they canbe actuated using a point and click device (such as a track ball ormouse). They can be actuated using hardware buttons, switches, ajoystick or keyboard, thumb switches or thumb pads, etc. They can alsobe actuated using a virtual keyboard or other virtual actuators. Inaddition, where the screen on which they are displayed is a touchsensitive screen, they can be actuated using touch gestures. Also, wherethe device that displays them has speech recognition components, theycan be actuated using speech commands.

A number of data stores have also been discussed. It will be noted theycan each be broken into multiple data stores. All can be local to thesystems accessing them, all can be remote, or some can be local whileothers are remote. All of these configurations are contemplated herein.

Also, the figures show a number of blocks with functionality ascribed toeach block. It will be noted that fewer blocks can be used so thefunctionality is performed by fewer components. Also, more blocks can beused with the functionality distributed among more components.

It will also be noted that the elements of FIG. 1 , or portions of them,can be disposed on a wide variety of different devices. Some of thosedevices include servers, desktop computers, laptop computers, tabletcomputers, or other mobile devices, such as palm top computers, cellphones, smart phones, multimedia players, personal digital assistants,etc.

FIG. 7 is a simplified block diagram of one illustrative example of ahandheld or mobile computing device that can be used as a user's orclient's hand held device 16, in which the present system (or parts ofit) can be deployed. For instance, a mobile device can be deployed inthe operator compartment of work machine 102 or as remote system 114.

FIG. 7 provides a general block diagram of the components of device 16that can run some components shown in FIG. 1 , that interacts with them,or both. In device 16, a communications link 13 is provided that allowsthe handheld device to communicate with other computing devices andunder some embodiments provides a channel for receiving informationautomatically, such as by scanning. Examples of communications link 13include allowing communication though one or more communicationprotocols, such as wireless services used to provide cellular access toa network, as well as protocols that provide local wireless connectionsto networks.

In other examples, applications can be received on a removable SecureDigital (SD) card that is connected to an interface 15. Interface 15 andcommunication links 13 communicate with a processor 17 (which can alsoembody processors or servers from previous FIGS.) along a bus 19 that isalso connected to memory 21 and input/output (I/O) components 23, aswell as clock 25 and location system 27.

I/O components 23, in one example, are provided to facilitate input andoutput operations. I/O components 23 for various embodiments of thedevice 16 can include input components such as buttons, touch sensors,optical sensors, microphones, touch screens, proximity sensors,accelerometers, orientation sensors and output components such as adisplay device, a speaker, and or a printer port. Other I/O components23 can be used as well.

Clock 25 illustratively comprises a real time clock component thatoutputs a time and date. It can also, illustratively, provide timingfunctions for processor 17.

Location system 27 illustratively includes a component that outputs acurrent geographical location of device 16. This can include, forinstance, a global positioning system (GPS) receiver, a LORAN system, adead reckoning system, a cellular triangulation system, or otherpositioning system. It can also include, for example, mapping softwareor navigation software that generates desired maps, navigation routesand other geographic functions.

Memory 21 stores operating system 29, network settings 31, applications33, application configuration settings 35, data store 37, communicationdrivers 39, and communication configuration settings 41. Memory 21 caninclude all types of tangible volatile and non-volatilecomputer-readable memory devices. It can also include computer storagemedia (described below). Memory 21 stores computer readable instructionsthat, when executed by processor 17, cause the processor to performcomputer-implemented steps or functions according to the instructions.Processor 17 can be activated by other components to facilitate theirfunctionality as well.

Examples of device 16 include, but are not limited to, a smart phone ortablet computer having a user interface display screen, such as a touchscreen or a pen-enabled interface that receives inputs from a pen orstylus. It can also use an on-screen virtual keyboard. Of course, itmight also be attached to a keyboard or other user input device througha suitable attachment mechanism, such as a wireless link or USB port,for instance. The computer can also illustratively receive voice inputsas well. Of course, of forms of devices 16 are possible.

FIG. 8 is one example of a computing environment in which elements ofFIG. 1 , or parts of it, (for example) can be deployed. With referenceto FIG. 8 , an example system for implementing some embodiments includesa computing device in the form of a computer 810. Components of computer810 may include, but are not limited to, a processing unit 820 (whichcan comprise processors or servers from previous FIGS.), a system memory830, and a system bus 821 that couples various system componentsincluding the system memory to the processing unit 820. The system bus821 may be any of several types of bus structures including a memory busor memory controller, a peripheral bus, and a local bus using any of avariety of bus architectures. Memory and programs described with respectto FIG. 1 can be deployed in corresponding portions of FIG. 8 .

Computer 810 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 810 and includes both volatile and nonvolatile media,removable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media is different from, anddoes not include, a modulated data signal or carrier wave. It includeshardware storage media including both volatile and nonvolatile,removable and non-removable media implemented in any method ortechnology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical disk storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to store the desired information and which canbe accessed by computer 810. Communication media may embody computerreadable instructions, data structures, program modules or other data ina transport mechanism and includes any information delivery media. Theterm “modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal.

The system memory 830 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as read only memory (ROM) 831and random access memory (RAM) 832. A basic input/output system 833(BIOS), containing the basic routines that help to transfer informationbetween elements within computer 810, such as during start-up, istypically stored in ROM 831. RAM 832 typically contains data and/orprogram modules that are immediately accessible to and/or presentlybeing operated on by processing unit 820. By way of example, and notlimitation, FIG. 8 illustrates operating system 834, applicationprograms 835, other program modules 836, and program data 837.

The computer 810 may also include other removable/non-removablevolatile/nonvolatile computer storage media. By way of example only,FIG. 8 illustrates a hard disk drive 841 that reads from or writes tonon-removable, nonvolatile magnetic media, an optical disk drive 855,and nonvolatile optical disk 856. The hard disk drive 841 is typicallyconnected to the system bus 821 through a non-removable memory interfacesuch as interface 840, and optical disk drive 855 is typically connectedto the system bus 821 by a removable memory interface, such as interface850.

Alternatively, or in addition, the functionality described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-programmable Gate Arrays(FPGAs), Application-specific Integrated Circuits (e.g., ASICs),Application-specific Standard Products (e.g., ASSPs), System-on-a-chipsystems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 8 , provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 810. In FIG. 8 , for example, hard disk drive 841 isillustrated as storing operating system 844, application programs 845,other program modules 846, and program data 847. Note that thesecomponents can either be the same as or different from operating system834, application programs 835, other program modules 836, and programdata 837.

A user may enter commands and information into the computer 810 throughinput devices such as a keyboard 862, a microphone 863, and a pointingdevice 861, such as a mouse, trackball or touch pad. Other input devices(not shown) may include a joystick, game pad, satellite dish, scanner,or the like. These and other input devices are often connected to theprocessing unit 820 through a user input interface 860 that is coupledto the system bus, but may be connected by other interface and busstructures. A visual display 891 or other type of display device is alsoconnected to the system bus 821 via an interface, such as a videointerface 890. In addition to the monitor, computers may also includeother peripheral output devices such as speakers 897 and printer 896,which may be connected through an output peripheral interface 895.

The computer 810 is operated in a networked environment using logicalconnections (such as a local area network—LAN, or wide area network—WANor a controller area network—CAN) to one or more remote computers, suchas a remote computer 880.

When used in a LAN networking environment, the computer 810 is connectedto the LAN 871 through a network interface or adapter 870. When used ina WAN networking environment, the computer 810 typically includes amodem 872 or other means for establishing communications over the WAN873, such as the Internet. In a networked environment, program modulesmay be stored in a remote memory storage device. FIG. 8 illustrates, forexample, that remote application programs 885 can reside on remotecomputer 880.

It should also be noted that the different examples described herein canbe combined in different ways. That is, parts of one or more examplescan be combined with parts of one or more other examples. All of this iscontemplated herein.

Example 1 is a mobile work machine comprising:

a frame;

a material loading system having a material receiving area configured toreceive material and an actuator configured to control the materialloading system to move the material receiving area relative to theframe; and

a control system configured to:

-   -   receive an indication of a detected object;    -   determine a location of the object relative to the material        loading system; and    -   generate a control signal that controls the mobile work machine        based on the determined location.

Example 2 is the mobile work machine of any or all previous examples,wherein the mobile work machine comprises a construction machine.

Example 3 is the mobile work machine of any or all previous examples,wherein the construction machine comprises a dump truck.

Example 4 is the mobile work machine of any or all previous examples,wherein the material receiving area comprises a bucket.

Example 5 is the mobile work machine of any or all previous examples,wherein the construction vehicle comprises one of an excavator or aloader.

Example 6 is the mobile work machine of any or all previous examples,wherein the material receiving area comprises a dump body.

Example 7 is the mobile work machine of any or all previous examples,wherein the actuator comprises a dumping actuator configured to move thedump body, and the control signal controls the dumping actuator.

Example 8 is the mobile work machine of any or all previous examples,wherein the control system comprises:

collision logic configured to receive a commanded movement from anoperator and determine whether the commanded movement will result incontact between the material loading system and the detected object; and

control logic configured to generate the control signal to prevent thecontact between the material loading system and the detected object.

Example 9 is the mobile work machine of any or all previous examples,wherein the control signal controls the dumping actuator to limitmovement of the material loading system.

Example 10 is the mobile work machine of any or all previous examples,and further comprising a set of ground engaging elements movablysupported relative to the frame, wherein the control signal controls atleast one of a propulsion system or a steering system that controls theset of ground engaging elements.

Example 11 is the mobile work machine of any or all previous examples,wherein the control signal controls a user interface mechanism toprovide an output to the user, the output indicating the location of theobject relative to the material loading system.

Example 12 is the mobile work machine of any or all previous examples,wherein the output comprises at least one of:

a display output;

an audible output; or

a haptic output.

Example 13 is the mobile work machine of any or all previous examples,and further comprising:

an object detection system configured to receive a signal from an objectdetection sensor and to detect the object based on the signal.

Example 14 is the mobile work machine of any or all previous examples,wherein the object detection sensor comprises an imaging sensor.

Example 15 is the mobile work machine of any or all previous examples,wherein the signal comprises a radio frequency (RF) signal.

Example 16 is a computer-implemented method of controlling a mobile workmachine, the method comprising:

receiving an indication of a detected object on a worksite;

determining a location of the object relative to a material loadingsystem of the mobile work machine, the material loading system having amaterial receiving area configured to receive material and an actuatorconfigured to control the material loading system to move the materialreceiving area relative to the frame; and

generating a control signal that controls the mobile work machine basedon the determined location.

Example 17 is the computer-implemented method of any or all previousexamples, wherein the mobile work machine comprises a constructionmachine, and further comprising:

receiving a commanded movement from an operator;

determining whether the commanded movement will result in contactbetween the material loading system and the detected object; and

generating the control signal to prevent the contact between thematerial loading system and the detected object.

Example 18 is a control system for a mobile work machine, the controlsystem comprising:

an object detection system configured to:

-   -   receive a signal from an object detection sensor and to detect        an object based on the signal; and    -   determine a location of the object relative to a material        loading system of the mobile work machine, the material loading        system having a material receiving area configured to receive        material and an actuator configured to control the material        loading system to move the material receiving area relative to        the frame;    -   collision logic configured to receive a commanded movement from        an operator and determine whether the commanded movement will        result in contact between the material loading system and the        detected object; and

control logic configured to generate the control signal to prevent thecontact between the material loading system and the detected object.

Example 19 is the control system of any or all previous examples,wherein the mobile work machine comprises a construction machine.

Example 20 is the control system of any or all previous examples,wherein the material receiving area comprises a dump body, and theactuator comprises a dumping actuator configured to move the dump body,and the control signal controls the dumping actuator.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed is:
 1. A mobile work machine comprising: a controllablesubsystem having a movable component that supports a material receiverconfigured to receive material; an actuator configured to move thecontrollable subsystem; and a control system configured to: determine alocation of a detected object relative to the mobile work machine;define a threshold distance based on an object type of the detectedobject; receive an indication of a commanded movement of the actuator tomove a portion of the controllable subsystem to a target position thatis within the threshold distance from the location of the detectedobject; and generate a control signal that controls the mobile workmachine based on the indication.
 2. The mobile work machine of claim 1,wherein the mobile work machine comprises a construction machine.
 3. Themobile work machine of claim 2, wherein the material receiver comprisesa bucket.
 4. The mobile work machine of claim 2, wherein theconstruction machine comprises a dump truck.
 5. The mobile work machineof claim 4, wherein the construction machine comprises one of anexcavator or a loader.
 6. The mobile work machine of claim 1, whereinthe material receiver comprises a dump body.
 7. The mobile work machineof claim 6, wherein the actuator comprises a dump actuator configured tomove the dump body, and the control signal controls the dump actuator.8. The mobile work machine of claim 1, wherein the control system isconfigured to: determine the object type by applying an objectclassifier to an image of the detected object.
 9. The mobile workmachine of claim 1, and further comprising a set of ground engagingelements movably supported relative to the frame, wherein the controlsignal controls at least one of a propulsion system or a steering systemthat controls the set of ground engaging elements.
 10. The mobile workmachine of claim 1, wherein the control signal controls a user interfacemechanism to provide an output to a user, the output indicating thelocation of the detected object relative to the mobile work machine, andwherein the output comprises at least one of: a display output; anaudible output; or a haptic output.
 11. The mobile work machine of claim1, and further comprising: an object detection system configured toreceive a signal from an object detection sensor and to detect thedetected object based on the signal.
 12. The mobile work machine ofclaim 11, wherein the object detection sensor comprises an imagingsensor.
 13. The mobile work machine of claim 11, wherein the signalcomprises a radio frequency (RF) signal.
 14. The mobile work machine ofclaim 1, wherein the control signal controls the actuator to limitmovement of the controllable subsystem.
 15. The mobile work machine ofclaim 14, wherein the control signal controls the actuator to preventcontact between the controllable subsystem and the detected object. 16.A computer-implemented method of controlling a mobile work machine, thecomputer-implemented method comprising: determining a location of adetected object relative to the mobile work machine; defining athreshold distance based on an object type of the detected object;receiving an indication of a commanded movement of an actuator to move aportion of a controllable subsystem to a target position that is withinthe threshold distance from the location of the detected object; andgenerating a control signal that controls the mobile work machine basedon the indication.
 17. The computer-implemented method of claim 16,wherein the mobile work machine comprises a construction machine, theactuator comprises a dump actuator configured to move a dump body, andgenerating the control signal comprises controlling the dump actuator.18. The computer-implemented method of claim 16, wherein generating thecontrol signal comprises: controlling at least one of a propulsionsystem or a steering system that controls a set of ground engagingelements of the mobile work machine.
 19. The computer-implemented methodof claim 16, wherein generating the control signal comprises:controlling a user interface mechanism to provide an output to a user,the output indicating the location of the detected object relative tothe mobile work machine, and wherein the output comprises at least oneof: a display output; an audible output; or a haptic output.
 20. Acontrol system for a mobile work machine, the control system comprising:an object detection system configured to: receive a signal from anobject detection sensor; detect an object based on the signal; anddetermine a location of the object relative to a controllable subsystemhaving an actuator configured to move a material receiver; collisionlogic configured to: define a threshold distance based on an object typeof the object; and receive an indication of a commanded movement of theactuator to move a portion of the controllable subsystem to a targetposition that is within the threshold distance from the location of theobject; and control logic configured to generate a control signal basedon the indication.